System and method of capturing physiological anomalies utilizing a vehicle seat

ABSTRACT

A vehicle system that includes a vehicle seat, wherein the vehicle seat includes a seat-back portion, a seat-bottom portion, a head rest portion, wherein the vehicle seat includes one or more acoustic sensors configured to retrieve an acoustic signal associated with a passenger seat, and a processor in communication with at least the one or more acoustic sensors, wherein the processor is programmed to identify an anomaly associated with the passenger utilizing the acoustic signal, wherein the anomaly is identified via pre-processing the acoustic signal and extracting one or more features associated with the acoustic signal in response to the pre-preprocessing and utilizing a classifier to classify one or more features associated with the acoustic signal as either a normal condition or the anomaly, and output a notification associated with the anomaly in response to identifying the anomaly.

TECHNICAL FIELD

The present disclosure relates to seats, such as those in a vehicle,which may utilize a microphone.

BACKGROUND

People use cars to commute to a variety of places. While someone is on acar seat and the car is moving, the person is usually static, and thisprovides a great opportunity to sense physiological parameters of theindividual. We propose to leverage this opportunity to sense a widerange of cardiovascular and respiratory issues, including but notlimited to heart murmur, heart arrythmia, coronary artery disease,coughing, sneezing, wheezing, shortness of breath, and asthma. In orderto do this, we propose to instrument a car seat with one or moremicrophone or acoustic sensors. The proposed work is not limited toinstrumenting the driver's seat only. It can be instrumented to eachseat of a car. Also, it can be instrumented in an infant/toddler carseat. It can also be instrumented to bus, truck, and airplane seats aswell. In addition, the chairs in other places including houses,commercial places, hospitals, airports, stadiums, and convention centerscan be instrumented and can be used for the proposed work.

SUMMARY

A first embodiment discloses a vehicle system that includes a vehicleseat, wherein the vehicle seat includes a seat-back portion, aseat-bottom portion, a head rest portion, wherein the vehicle seatincludes one or more acoustic sensors configured to retrieve an acousticsignal associated with a passenger seat, and a processor incommunication with at least the one or more acoustic sensors, whereinthe processor is programmed to identify an anomaly associated with thepassenger utilizing the acoustic signal, wherein the anomaly isidentified via pre-processing the acoustic signal and extracting one ormore features associated with the acoustic signal in response to thepre-preprocessing and utilizing a classifier to classify one or morefeatures associated with the acoustic signal as either a normalcondition or the anomaly, and output a notification associated with theanomaly in response to identifying the anomaly.

A second embodiment discloses a vehicle system that includes a vehicleseat, wherein the vehicle seat includes a seat-back portion, aseat-bottom portion, wherein the vehicle seat includes one or moreacoustic sensors configured to retrieve an acoustic signal associatedwith a passenger seat, and a processor in communication with at leastthe one or more acoustic sensors, wherein the processor is programmed toclassify the acoustic signal as an anomaly or normal condition, whereinthe classifying is accomplished pre-processing the acoustic signal andextracting one or more features associated with the acoustic signal inresponse to the pre-preprocessing and utilizing a classifier to classifyone or more features associated with the acoustic signal as the normalcondition or the anomaly, and output a notification associated with theanomaly in response to identifying the anomaly.

A third embodiment discloses a system that includes a processor incommunication with at least one or more acoustic sensors located in aseat, wherein the processor is programmed to identify an anomalyassociated with the passenger utilizing the acoustic signal, wherein theanomaly is identified via pre-processing the acoustic signal andextracting one or more features associated with the acoustic signal inresponse to the pre-preprocessing and utilizing one or more featuresassociated with the acoustic signal as the anomaly, and output anotification associated with the anomaly in response to identifying theanomaly.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 discloses an embodiment of a schematic diagram as related to avehicle seat that includes acoustic sensors.

FIG. 2A discloses a first embodiment of a vehicle seat including oneunit.

FIG. 2B disclose a second embodiment of a vehicle seat including twounits.

FIG. 2C discloses a third embodiment of a vehicle seat including threeunits.

FIG. 2D disclose a fourth embodiment with a rectangular array of fourunits.

FIG. 2E discloses a fifth embodiment with a rectangular array of fiveunits.

FIG. 2F discloses a sixth embodiment with a circular array with sixunits.

FIG. 2G discloses a seventh embodiment with a circular array with sevenunits.

FIG. 3 discloses a flowchart of a computing pipeline for detectingphysiological anomaly.

FIG. 4 discloses an alternative flowchart of a computing pipeline fordetecting physiological anomaly according to a second embodiment.

DETAILED DESCRIPTION

Embodiments of the present disclosure are described herein. It is to beunderstood, however, that the disclosed embodiments are merely examplesand other embodiments can take various and alternative forms. Thefigures are not necessarily to scale; some features could be exaggeratedor minimized to show details of particular components. Therefore,specific structural and functional details disclosed herein are not tobe interpreted as limiting, but merely as a representative basis forteaching one skilled in the art to variously employ the embodiments. Asthose of ordinary skill in the art will understand, various featuresillustrated and described with reference to any one of the figures canbe combined with features illustrated in one or more other figures toproduce embodiments that are not explicitly illustrated or described.The combinations of features illustrated provide representativeembodiments for typical applications. Various combinations andmodifications of the features consistent with the teachings of thisdisclosure, however, could be desired for particular applications orimplementations.

People may use vehicles for various scenarios, such as for commuting towork and to go to many other places on a regular basis. While someone ison a car seat, the body movement is constrained while the car is inmotion. We leverage this constrained setup to opportunistically sensethe physiological parameters to detect health anomalies. In particular,within the scope of this invention, we instrument car seats withmicrophones or acoustic sensors that listens to heart sounds (audibleand infrasound) and other physiologically meaningful sounds emanatingfrom the body to detect a wide range of health issues, including but notlimited to heart murmur, heart arrhythmia, coronary artery disease, andother cardiovascular diseases. It can also be useful for detectingcoughing, sneezing, wheezing, shortness of breath, asthma, and otherrespiratory diseases.

FIG. 1 discloses a schematic. The vehicle seat 101 may include variouscomponents to help listen for physiological anomalies. The vehicle seat101 may be any type of seat, such as an infant car seat, toddler carseat, bus seat, train seat, airplane seat, motorcycle seat, etc. Forexample, the vehicle seat 101 may include one or more acoustic sensors103 a, 103 b, 103 c. As explained further below, the acoustic sensorsmay be arranged in any manner in the vehicle seat. The acoustic sensors103 may be arranged under a perforation(s) of the seat. In oneembodiment, the acoustic sensors 103 may be in a seat-back portion ofthe vehicle seat 101, however, any area of the vehicle seat may suffice.The acoustic sensors 103 a, 103 b, 103 c, may be any type of sensor thatcan pick up any sound or acoustic signal, such as a microphone, sonar,Thickness-Shear Mode resonator, surface acoustic wave (SAW) sensor,Shear-Horizontal Acoustic Plate Mode (SH APM) sensor, Flexural PlateWave (FPW) sensor, etc.

Conversion of acoustic energy to electrical energy and electrical energyto acoustic energy is known in the art. Conversion of digital signals toanalog signals and conversion of analog signals to digital signals isalso known. Processing digital representations of energy and analogrepresentations of energy either in hardware or by software directedcomponents is also well known.

Separately, there may be an on/off 105 button located in the vehiclesystem. The one/off 105 switch may be utilized to turn off power to thevehicle seat or at least the acoustic sensors. If a switch is ON, it hasa value of 1 or zero (0) and may activate power to its components. If itis OFF it may have no value, so it may be represented with an “0.” Thesame may be true for the state of the switch being ON or OFF. When youare looking at a schematic and reading the values for each switchposition, you are counting from left to right. Additionally, a powersource 107 may be located in the vehicle seat or connecting to thevehicle seat. The power source 107 may provide power to the one or moreacoustic sensors 103, additionally to other components. The power source107 may be connected to a vehicle alternator, battery, or any otherenergy source (e.g. solar, etc.).

The control circuit 109 may include a controller or a processor. Thecontrol circuit may include both volatile and non-volatile storage.Non-volatile storage may be included one or more persistent data storagedevices such as a hard drive, optical drive, tape drive, non-volatilesolid-state device, cloud storage or any other device capable ofpersistently storing information. The processor may include one or moredevices selected from high-performance computing (HPC) systems includinghigh-performance cores, microprocessors, micro-controllers, digitalsignal processors, microcomputers, central processing units, fieldprogrammable gate arrays, programmable logic devices, state machines,logic circuits, analog circuits, digital circuits, or any other devicesthat manipulate signals (analog or digital) based on computer-executableinstructions residing in memory. Memory may include a single memorydevice or a number of memory devices including, but not limited to,random access memory (RAM), volatile memory, non-volatile memory, staticrandom access memory (SRAM), dynamic random access memory (DRAM), flashmemory, cache memory, or any other device capable of storinginformation.

Processor may be configured to read into memory and executecomputer-executable instructions residing in non-volatile storage andembodying one or more algorithms and/or methodologies of one or moreembodiments. Non-volatile storage may include one or more operatingsystems and applications. Non-volatile storage may store compiled and/orinterpreted from computer programs created using a variety ofprogramming languages and/or technologies, including, withoutlimitation, and either alone or in combination, Java, C, C++,C#,Objective C, Fortran, Pascal, Java Script, Python, Perl, and PL/SQL.

Upon execution by processor, the computer-executable instructions ofnon-volatile storage may cause control system to implement one or moreof the algorithms and/or methodologies as disclosed herein. Non-volatilestorage may also include machine learning (ML) data (including dataparameters) supporting the functions, features, and processes of the oneor more embodiments described herein.

The program code embodying the algorithms and/or methodologies describedherein is capable of being individually or collectively distributed as aprogram product in a variety of different forms. The program code may bedistributed using a computer readable storage medium having computerreadable program instructions thereon for causing a processor to carryout aspects of one or more embodiments. Computer readable storage media,which is inherently non-transitory, may include volatile andnon-volatile, and removable and non-removable tangible media implementedin any method or technology for storage of information, such ascomputer-readable instructions, data structures, program modules, orother data. Computer readable storage media may further include RAM,ROM, erasable programmable read-only memory (EPROM), electricallyerasable programmable read-only memory (EEPROM), flash memory or othersolid state memory technology, portable compact disc read-only memory(CD-ROM), or other optical storage, magnetic cassettes, magnetic tape,magnetic disk storage or other magnetic storage devices, or any othermedium that can be used to store the desired information and which canbe read by a computer. Computer readable program instructions may bedownloaded to a computer, another type of programmable data processingapparatus, or another device from a computer readable storage medium orto an external computer or external storage device via a network.

Computer readable program instructions stored in a computer readablemedium may be used to direct a computer, other types of programmabledata processing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions thatimplement the functions, acts, and/or operations specified in theflowcharts or diagrams. In certain alternative embodiments, thefunctions, acts, and/or operations specified in the flowcharts anddiagrams may be re-ordered, processed serially, and/or processedconcurrently consistent with one or more embodiments. Moreover, any ofthe flowcharts and/or diagrams may include more or fewer nodes or blocksthan those illustrated consistent with one or more embodiments.

The processes, methods, or algorithms can be embodied in whole or inpart using suitable hardware components, such as Application SpecificIntegrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs),state machines, controllers or other hardware components or devices, ora combination of hardware, software and firmware components.

The visualization output 111 may include a display. The display mayinclude any vehicle display, such as a multimedia display. The systemmay also include a human-machine interface (HMI) display. The HMIdisplay may include any type of display within a vehicle cabin. Such HMIdisplay may include a dashboard display, navigation display, multimediadisplay, heads-up display, thin-film transistor liquid-crystal display(TFT LCD), etc. The HMI display may also be connected to speakers tooutput sound related to commands or the user interface of the vehicle.The HMI display may be utilized to output various commands orinformation to occupants (e.g. driver or passengers) within the vehicle.For example, in an automatic braking scenario, the HMI display maydisplay a message that the vehicle is prepared to brake and providefeedback to the user regarding the same. The HMI display may utilize anytype of monitor or display utilized to display relevant information tothe occupants. In addition to providing visual indications, the displaymay also be configured to receive user input via a touch-screen, userinterface buttons, etc. The display may be configured to receive usercommands indicative of various vehicle controls such as audio-visualcontrols, autonomous vehicle system controls, certain vehicle features,cabin temperature control, etc. The controller may receive such userinput and in turn command a relevant vehicle system of the component toperform in accordance with the user input.

FIG. 2A discloses a first embodiment of a vehicle seat including oneunit. FIG. 2B disclose a second embodiment of a vehicle seat includingtwo units. In such an embodiment, the acoustic sensors or microphonesmay be located in the seat-back portion of the vehicle seat. In theembodiment of FIG. 2A, only one acoustic sensor may be utilized. In FIG.2B, two acoustic sensors may be utilized. The sensors may be in a singleseat (e.g., driver seat or passenger seat), or in all seats, or anycombination thereto. In FIG. 2B, the two sensors may be lined up in alinear array with two units. FIG. 2C discloses a third embodiment of avehicle seat including three units. FIG. 2D disclose a fourth embodimentwith a rectangular array of four units. FIG. 2E discloses a fifthembodiment with a rectangular array of five units. Such an array may bea rectangular array. The additional sensors in the rectangular array maybe able to detect acoustic waves emitted form a user. In one embodiment,the acoustic waves may be penetrated through perforations of the seat.FIG. 2F discloses a sixth embodiment with a circular array with sixunits. FIG. 2G discloses a seventh embodiment with a circular array withseven units. Such an array may be a circular array. The additionalsensors in a circular array may be able to detect acoustic waves emittedform a user. In one embodiment, the acoustic waves may be penetratedthrough perforations of the seat.

FIG. 3 discloses a flowchart of a computing pipeline for detectingphysiological anomaly. In such an embodiment, each physiological anomalymay be represented as a class. The system may have a class for normalhealth conditions as well. The system may capture time-frequency domainacoustic features (e.g., Mel-spectorgram) from each window of a sensingperiod and then classify the window utilizing a classifier, e.g., anSVM, random forest, or a multilayer perceptron. After the classificationis performed, the classification result may show whether the person hasa normal health condition or has a physiological anomaly. Theclassification result may also describe the type of anomaly, e.g., heartarrhythmia based on the class with the highest confidence score. At step301, the system may collect an acoustic signal. The acoustic signal maycome from a one or more acoustic sensors. The acoustic sensors may allbe the same type of sensor or may be a mix of various types of sensors.The acoustic sensor may be utilized to collect information related tothe user to identify any strange behavior or patterns. The sensors maylisten to audio at a specific sampling rate. At step 303, the system mayretrieve the acoustic signals for pre-processing at a circuit and/ormicrophones. At step 305, the pre-processed acoustic signal may be fedto a controller or another circuit for feature extraction. The featureextraction may be utilized to retrieve relevant audio features foranalysis. The controller or processor may utilize time-frequency domainfeatures, e.g., MFCC, SoundNet, or similar neural network basedarchitecture or any combination to extract the relevant features. Atstep 307, the system may classify the acoustic signals. The system mayutilize a classifier to classify the audio event. For suchclassification, an SVM, random forest, or multiplayer perceptronclassifier may be used. The classifiers may classify the audio event tothe events of interest, including but not limited to a normal healthcondition, heart arrhythmia, coronary artery disease, coughing,sneezing, wheezing, shortness of breath, asthma, and othercardiovascular and respiratory disease. Prior to this classification,the classifier is trained utilizing collected data covering all classesof interest. At step 309, the system may utilize time-seriesaggregation. During the time series aggregation, the health conditiondetected throughout the entire sensing period may be aggregated. Forexample, the system may calculate how many times a person showed a caseof heart arrhythmia and other symptoms of illness. It may be used toimprove the detection as well. For example, the time-series aggregationprocess may discard sporadic detection incorrectly classified events byutilizing a majority voting. It may also be utilized to provide asummary of the health condition detected from the entire sensing period.At step 311, the system may utilize visualization to output theaggregated information. For example, the information may be output atthe display of the car, an app of a smart phone, or a display. If thehealth condition is critical, it can be shown immediately withoutconducting the time-series aggregation, such as in step 309.

FIG. 4 discloses an alternative flowchart of a computing pipeline fordetecting physiological anomaly according to a second embodiment. Insuch an embodiment, the system may not train a classifier for eachphysiological anomaly. Instead, the system may focus on learning thenormal health condition and detect deviation from the normalphysiological parameters. The steps for detecting physiologicalanomalies may use this approach similar to the previous embodiment.However, during the classification phase may not exist. At step 401, thesystem may collect an acoustic signal. The acoustic signal may come froma one or more acoustic sensors. The acoustic sensors may all be the sametype of sensor or may be a mix of various types of sensors. The acousticsensor may be utilized to collect information related to the user toidentify any strange behavior or patterns. At step 403, the system mayretrieve the acoustic signals for pre-processing at a circuit. and/ormicrophones. At step 405, the pre-processed acoustic signal may be fedto a controller or another circuit for feature extraction. The featureextraction may be utilized to retrieve relevant audio features foranalysis. The controller or processor may utilize time-frequency domainfeatures, e.g., AfFCC, SoundNet, or similar neural network basedarchitecture or any combination to extract the relevant features. Atstep 407, the system may determine if any anomalies are detected. Thesystem may utilize an anomaly detector to detect deviations from normalhealth conditions using the features extracted. If the deviation isbeyond a pre-defined threshold, then it may report this as an anomaly.Prior to this, features of normal health condition and anomalous healthconditions are fed to a clustering algorithm to tune its parameters tocluster normal health condition from the rest. For clustering algorithm,DBScan, K-means, Agglomerative clustering, spectral clustering, or otherclustering algorithm can be utilized. The system can also apply otheranomaly detection algorithms on acoustic features. For example,one-class SVM, Gaussian Mixture Model, and auto-encoders may beutilized. At step 409, the system may utilize time-series aggregation.During the time series aggregation, the health condition detectedthroughout the entire sensing period may be aggregated. For example, thesystem may calculate how many times a person showed a case of heartarrhythmia and other symptoms of illness. It may be used to improve thedetection as well. For example, the time-series aggregation process maydiscard sporadic detection incorrectly classified events by utilizing amajority voting. It may also be utilized to provide a summary of thehealth condition detected from the entire sensing period. At step 411,the system may utilize visualization to output the aggregatedinformation. For example, the information may be output at the displayof the car, an app of a smart phone, or a display. If the healthcondition is critical, it can be shown immediately without conductingthe time-series aggregation, such as in step 409.

While exemplary embodiments are described above, it is not intended thatthese embodiments describe all possible forms encompassed by the claims.The words used in the specification are words of description rather thanlimitation, and it is understood that various changes can be madewithout departing from the spirit and scope of the disclosure. Aspreviously described, the features of various embodiments can becombined to form further embodiments of the invention that may not beexplicitly described or illustrated. While various embodiments couldhave been described as providing advantages or being preferred overother embodiments or prior art implementations with respect to one ormore desired characteristics, those of ordinary skill in the artrecognize that one or more features or characteristics can becompromised to achieve desired overall system attributes, which dependon the specific application and implementation. These attributes caninclude, but are not limited to cost, strength, durability, life cyclecost, marketability, appearance, packaging, size, serviceability,weight, manufacturability, ease of assembly, etc. As such, to the extentany embodiments are described as less desirable than other embodimentsor prior art implementations with respect to one or morecharacteristics, these embodiments are not outside the scope of thedisclosure and can be desirable for particular applications.

What is claimed is:
 1. A vehicle system, including: a vehicle seat,wherein the vehicle seat includes a seat-back portion, a seat-bottomportion, wherein the vehicle seat includes one or more acoustic sensorsconfigured to retrieve an acoustic signal associated with a passengerseat; and a processor in communication with at least the one or moreacoustic sensors, wherein the processor is programmed to: identify ananomaly associated with the passenger utilizing the acoustic signal,wherein the anomaly is identified via pre-processing the acoustic signaland extracting one or more features associated with the acoustic signalin response to the pre-preprocessing and utilizing a classifier toclassify one or more features associated with the acoustic signal aseither a normal condition or the anomaly; and output a notificationassociated with the anomaly in response to identifying the anomaly. 2.The vehicle system of claim 1, wherein the one or more features areextracted utilizing Mel Frequency Cepstral Coefficients (MFCC), SoundNetconvolutional neural network (CNN), time domain features, or frequencydomain features.
 3. The vehicle system of claim 1, wherein the processoris further programmed to output information associated with the normalcondition in response to both identifying the normal condition andutilizing a time-series aggregation.
 4. The vehicle system of claim 1,wherein the classifier is trained to identify the anomaly via machinelearning.
 5. The vehicle system of claim 1, wherein the seat-backportion of the seat includes two acoustic sensors aligned in an arrayfashion.
 6. The vehicle system of claim 1, wherein the seat-back portionof the seat includes three acoustic sensors aligned in an array fashion.7. The vehicle system of claim 1, wherein the seat-back portion of theseat includes four acoustic sensors aligned in a rectangular arrayfashion.
 8. The vehicle system of claim 1, wherein the seat-back portionof the seat includes acoustic sensors aligned in a circular arrayfashion.
 9. A vehicle system, including: a vehicle seat, wherein thevehicle seat includes a seat-back portion, a seat-bottom portion,wherein the vehicle seat includes one or more acoustic sensorsconfigured to retrieve an acoustic signal associated with a passengerseat; and a processor in communication with at least the one or moreacoustic sensors, wherein the processor is programmed to: classify theacoustic signal as an anomaly or normal condition, wherein theclassifying is accomplished pre-processing the acoustic signal andextracting one or more features associated with the acoustic signal inresponse to the pre-preprocessing and utilizing a classifier to classifyone or more features associated with the acoustic signal as the normalcondition or the anomaly; and output a notification associated with theanomaly in response to identifying the anomaly.
 10. The vehicle systemof claim 9, wherein the one or more features are extracted utilizing MelFrequency Cepstral Coefficients (MFCC), SoundNet convolutional neuralnetwork (CNN), time domain features, or frequency domain features. 11.The vehicle system of claim 9, wherein the processor is furtherprogrammed to output information associated with the normal condition inresponse to both identifying the normal condition and utilizing atime-series aggregation.
 12. The vehicle system of claim 9, wherein theclassifier is trained to identify the anomaly via machine learning. 13.The vehicle system of claim 9, wherein the one or more acoustic sensorsis a microphone.
 14. The vehicle system of claim 1, wherein theseat-back portion of the seat includes the one or more acoustic sensorsaligned in a circular array fashion.
 15. The vehicle system of claim 1,wherein the seat-back portion of the seat includes one or more acousticsensors aligned in a rectangular array fashion.
 16. A system, including:a processor in communication with at least one or more acoustic sensorslocated in a seat, wherein the processor is programmed to: identify ananomaly associated with the passenger utilizing the acoustic signal,wherein the anomaly is identified via pre-processing the acoustic signaland extracting one or more features associated with the acoustic signalin response to the pre-preprocessing and utilizing one or more featuresassociated with the acoustic signal as the anomaly; and output anotification associated with the anomaly in response to identifying theanomaly.
 17. The system of claim 16, wherein the seat is a non-vehicleseat.
 18. The system of claim 16, wherein the processor is furtherprogrammed to output information associated with the normal condition inresponse to both identifying the normal condition and utilizing atime-series aggregation.
 19. The system of claim 16, wherein theprocessor is remote from the seat.
 20. The system of claim 1, whereinthe seat includes a child-car seat.